{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  为什么要引入核函数？（10分）<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;咋平时遇到的问题中样本点不宜一定是线性可分的，这就需要引入和函数来解决更高维空间下样本点的分割，可以将原始空间映射到一个更高维特征空 间，使得在这个特征空间数据线性可分。<br>\n",
    "&emsp;&emsp;核函数的具体形式只是将在对偶问题中推导出的SVM模型中的二维空间中的点击向量<**x_i**,**x**>,换成了高维空间中的点积，可写成核(kernel)的形式如下：常用的核函数是RBF核<br>\n",
    "![](http://qa7l81fun.bkt.clouddn.com/808.jpg)<br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 给出合页损失的数学形式并画出图形。（10分）<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;对于添加了松弛变量&xi;的SVM模型，就得到了软间隔最大化的C-SVM模型：<br>\n",
    "\n",
    "![804](http://qa7l81fun.bkt.clouddn.com/804.jpg)<br>\n",
    "&emsp;&emsp;函数图像如下图：<br>\n",
    "![801](http://qa7l81fun.bkt.clouddn.com/801.jpg)<br>\n",
    "&emsp;&emsp;对于该模型的理解：y(wx+b)的绝对值代表样本距离决策边界的远近程度。y(wx+b)的绝对值越大，表示样本距离决策边界越远。当y(wx+b)>0时，y(wx+b)的绝对值越大表示决策边界对样本的区分度越好；当y(wx+b)<0时，y(wx+b)的绝对值越大表示决策边界对样本的区分度越差<br>\n",
    "&emsp;&emsp;所以当样本被正确分类且函数间隔大于1时，合页损失才是0，否则损失是1-y(wx+b)。<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 什么是支持向量？为什么SVM中只有一小部分的训练样本是支持向量（稀疏的）？（10分）<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;对于SVM的KKT条件中的对偶互补条件：<br>\n",
    "![805](http://qa7l81fun.bkt.clouddn.com/805.jpg)<br>\n",
    "&emsp;&emsp;对每个训练样本点当满足:<br>\n",
    "![](http://qa7l81fun.bkt.clouddn.com/806.jpg)<br>\n",
    "&emsp;&emsp;这些样本点被称为支持向量,如下图所示，所谓支持向量的样本点就是指出现在两条间隔线上或者在两条间隔线内的点，或者像图中所示的跑到红色区域的黑点或者跑到黑色区域的红点都是支持向量。所以说支持向量是影响决策边界的数据点，凡是对决策边界有影响作用的样本点都可以作为支持向量。<br>\n",
    "&emsp;&emsp;所以说加进来的点只有影响决策面的样本点才是支持向量，这也是支持向量稀疏的原因。<br>\n",
    "![](http://qa7l81fun.bkt.clouddn.com/807.jpg)<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 决策树中特征分裂的准则有哪些？（10分）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;决策树的分裂准则决定了树的大小，包括树的宽度与深度。常用的分裂准则有信息增益、信息增益率、CART。在决策树的分裂过程中，每一支的属性可以重复出现，也就是说一个属性可以在不同层使用。<br>\n",
    "&emsp;&emsp;ID3选择信息增益最大的特征进行分裂，他是通过先计算每一个特征的熵（特征的混乱程度），然后再用系统熵与每一个特征的熵做差，选择系统熵值下降最大的作为分裂依据。<br>\n",
    "&emsp;&emsp;而信息增益率它考虑到了特征自身内部的熵值。这也是对信息增益的不足进行了改进，因为有时候会遇到一些样本，他的某些特征不影响系统的决策，反而成为了数据分类的噪声。就比如数据的ID,每个数据的ID都是不一样的，所以他熵值为0，所以他的信息增益成为最大值，系统会以他为根节点作为分类标准，从而失去了分裂的意义，而像这类特征它本身内部的熵是很大的。所以，用信息增益这个值来除以它本身内部的熵来降低噪声的影响。<br>\n",
    "&emsp;&emsp;CART：基尼系数不纯度（GiNi），Gini系数和熵的思想是一样的，也是反应了特征的不纯度，但他的计算方法要比熵的简单，，无需log，计算更快。另外ID3和C4.5可以生成多分叉的决策树，而CART只能是二叉树。<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SVM模型并没有概率解释。为了使SVM模型能输出概率，我们应该设置哪个参数？（10分）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;在SVM模型中，scikit-learn中提供了SVC，其中predict_log_proba(X)和predict_proba(X)两个方法返回了X对应的预测值（每个类别对应的概率的log值），只需要将参数probability被设为True。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 采用train_test_split，从将数据集中随机抽取10000条记录，用于下述作业中模型的训练（原始数据集太大，剩余数据抛弃）。（10分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "dpath = './data/'\n",
    "\n",
    "train1 = pd.read_csv(\"Otto_FE_train_org.csv\")\n",
    "train2 = pd.read_csv(\"Otto_FE_train_tfidf.csv\")\n",
    "\n",
    "#去掉多余的id，并且将两个训练数据串联\n",
    "train2 = train2.drop([\"id\", \"target\"], axis = 1)\n",
    "train = pd.concat([train1, train2], axis = 1, ignore_index = False)\n",
    "train.head()\n",
    "\n",
    "del train1\n",
    "del train2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#将类别字符串变成数字\n",
    "y_train = train['target']\n",
    "x_train = train.drop([\"id\", \"target\"], axis = 1)\n",
    "\n",
    "#保存特征名称（用于可视化）\n",
    "feat_names = x_train.columns\n",
    "\n",
    "#将数据稀疏化\n",
    "from scipy.sparse import csr_matrix\n",
    "x_train = csr_matrix(x_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "#训练样本6w+,交叉验证greatCV太慢，用train_test_split(调优)估计模型性能\n",
    "#SVM对大样本数据集支持不太好\n",
    "from sklearn.model_selection import train_test_split\n",
    "x_train_part, x_val, y_train_part, y_val = train_test_split(x_train, y_train, random_state = 0, train_size = 10000)#分成训练集和校验集，这里10000的数据用于训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 186)\n"
     ]
    }
   ],
   "source": [
    "print(x_train_part.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 根据6中得到的训练数据，训练线性SVM，并对超参数（正则惩罚项、C）进行超参数调优，并分析不同C对应的训练误差和交叉验证得到的测试误差的变化趋势。（20分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
       "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
       "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
       "     verbose=0)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#默认参数的SVC（缺省参数）\n",
    "\n",
    "from sklearn.svm import LinearSVC\n",
    "#LinearSVC不能得到每类的概率（只有predict函数，没有predict_proba函数），在Otto数据集要求输出每类的概率\n",
    "#生成学习器实例\n",
    "SVC1 = LinearSVC()\n",
    "\n",
    "#训练模型\n",
    "SVC1.fit(x_train_part, y_train_part)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy is: 0.7596283588418983\n",
      "Classification report for classifier LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
      "     verbose=0):\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "    Class_1       0.63      0.30      0.41      1615\n",
      "    Class_2       0.65      0.86      0.74     13479\n",
      "    Class_3       0.51      0.33      0.40      6706\n",
      "    Class_4       0.72      0.14      0.23      2267\n",
      "    Class_5       0.94      0.96      0.95      2312\n",
      "    Class_6       0.93      0.93      0.93     11883\n",
      "    Class_7       0.71      0.61      0.65      2399\n",
      "    Class_8       0.84      0.92      0.88      7077\n",
      "    Class_9       0.79      0.86      0.83      4140\n",
      "\n",
      "avg / total       0.75      0.76      0.74     51878\n",
      "\n",
      "\n",
      "Confusion matrix:\n",
      "[[  487    64    14     8     8   140    48   382   464]\n",
      " [   15 11626  1435    61    74    43   102    68    55]\n",
      " [    5  4209  2223    26     9    16   155    37    26]\n",
      " [    0  1356   386   308    17   114    62    17     7]\n",
      " [    3    71     4     0  2224     3     0     4     3]\n",
      " [   71   118    24    12     4 11006   166   270   212]\n",
      " [   57   260   193     7    18   171  1462   200    31]\n",
      " [   72    84    35     2     6   207    51  6494   126]\n",
      " [   69    73     6     5    10   158    23   218  3578]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "#训练好之后，就可以到校验集上去预测\n",
    "y_predict = SVC1.predict(x_val)\n",
    "print(\"accuracy is:\", accuracy_score(y_val, y_predict))\n",
    "print(\"Classification report for classifier %s:\\n%s\\n\"%(SVC1, classification_report(y_val, y_predict)))\n",
    "print(\"Confusion matrix:\\n%s\" % confusion_matrix(y_val, y_predict))#打印混淆矩阵(样本分类的个数，对角线是分对的)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#线性SVM正则参数调优\n",
    "#单组超参数情况，模型在训练集上训练，在校验集上的测试的测试性能\n",
    "def fit_grid_point_Linear(C, x_train, y_train, x_val, y_val):\n",
    "    \n",
    "    # 在训练集上训练SVC\n",
    "    SVC2 =  LinearSVC( C = C)\n",
    "    SVC2 = SVC2.fit(x_train, y_train)\n",
    "    \n",
    "    # 在校验集上返回accuracy\n",
    "    accuracy = SVC2.score(x_val, y_val)\n",
    "    \n",
    "    print(\"C= {} : accuracy= {} \" .format(C, accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "code_folding": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 0.1 : accuracy= 0.750183121939936 \n",
      "C= 1.0 : accuracy= 0.7596283588418983 \n",
      "C= 10.0 : accuracy= 0.7618836501021627 \n",
      "C= 100.0 : accuracy= 0.7584717992212499 \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No handles with labels found to put in legend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 1000.0 : accuracy= 0.6285901538224296 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmUVPWd9/H3h90FAaGNyCKYp9XgEpeOGs2ixhiMjxKiUdGoCBQzkzHPZCbxGZOZM5kxZ86Ymcw4S5w8UdSIGxLciDEhjFvMuDYRRSAo4kIHowRFg4ps3+ePe3u66IWqhr51q6o/r3PqUPfWr6q+XdD14f5+9/5+igjMzMx2pE/eBZiZWfVzWJiZWUkOCzMzK8lhYWZmJTkszMysJIeFmZmV5LAwM7OSHBZmZlaSw8LMzErql3cBPWXEiBExbty4vMswM6spixYt+n1ENJRqVzdhMW7cOJqbm/Muw8yspkh6pZx27oYyM7OSHBZmZlaSw8LMzEqqmzELM7PebvPmzbS0tLBx48YOjw0aNIjRo0fTv3//nXpth4WZWZ1oaWlh8ODBjBs3Dkn/sz8iWLduHS0tLYwfP36nXtvdUGZmdWLjxo0MHz58u6AAkMTw4cM7PeIol8PCzKyOtA+KUvvLlWk3lKSJwL8BfYFZEXFlu8evAk5KN3cH9omIoeljY4FZwBgggM9HxMtZ1mtWrq1b4d13224bNux4e8AAGDas89uQIdC3b94/kdmOZRYWkvoCVwOfBVqApyTNj4hlrW0i4s+L2n8VOLLoJWYDfx8RCyXtCWzLqlarT9u2wfvvl/dl3t3tXTia70CCvfbqGCJ77911wDhorNKyPLI4BlgZEasAJM0BJgHLumg/Bfh22nYC0C8iFgJExIYM67QcRcAHH/T8l/mGDfDee92rpX9/2GOP5Lbnnm33hw+H/fdv227/eKnt3XeHTZvgrbfKvy1b1nb/gw92XPeQIaVDpbPb0KEOmnoUEZ12OUXELr1ulmExClhdtN0CHNtZQ0n7A+OBB9JdBwLrJd2Z7v8v4PKI2JpduVaOCHj1Vfj973vuy31bN44Z+/Tp/Mt5yBDYb7+d+zJvvQ0YkN3n1q9fEhqjRnX/ue+/D2++mU3QdHZEU27Q9PO5lFVn0KBBrFu3rsMgd+vZUIMGDdrp187yr7uz0ZSuou08YF5RGPQDPknSLfUqcDswFbhuuzeQZgIzAcaOHbvrFdt23nsPnnsOnnmm7fbss/DOO6Wf2/6LuPXLecSInfsyb90eODDptulNdtstCZmdDZruHNH85jdt90t1tQ0e3L2Aae1Wc9BkZ/To0bS0tLB27doOj7VeZ7GzsvwrayEZnG41GljTRdvzgD9t99yni7qw7gaOo11YRMQ1wDUATU1Nu3aM1YtFwJo124fCM8/A88+3/a9/8GA4/HD48peTP0eO7PrLfbfdkiMAy99uuyW3/fbr/nM3buxe0Dz/fNv999/f8WuXCpozzkj+nVn39O/ff6evoygly7B4CmiUNB74LUkgnN++kaSDgGHAY+2eO0xSQ0SsBU4GPKVsD9i0CZYvT8Jg8eK2YFi3rq3NuHHw0Y/COeckf370ozB+vAOgtxk0KPlPwciR3X/uBx90L2heeKHt/nvvwZw5yVFsbzuKrGaZhUVEbJF0KbCA5NTZ6yNiqaQrgOaImJ82nQLMiaLRl4jYKukbwP1KOt4WAddmVWu9Wru249HC8uWweXPy+KBBcOihMHlyWygcfnjS/2+2KwYOhH33TW7ddc018Ed/BE88Accd1/O12c7Rro6QV4umpqboretZbN2adAEUh8LixfDaa21t9tuvLRBab42N7ju26vOHPyRHM+eeC9ddV7q97RpJiyKiqVQ7f1XUmLffTg7Pi7uQnnuubTCyXz+YMAFOOWX7YGgouQ6WWXUYPBimTIFbb4WrrkrO2LL8OSyq1LZt8NJLHbuRXn65rc3w4UkQfOUrbaHwkY9kewqoWSUUCjBrFtx2W9IlZflzN1QVePfd7U9RXbwYlixJDschGVg+8MCO3Uj77ecBQKtPEXDEEcmFkjX6a10z3A1VhSKgpaXj0cILLySPQXLIffjhcNFFbaFw6KHJBV1mvYWUHF189avw61/DUUflXZE5LDLywQfJlbTtg+HNN9vajB+fhMH557cFw7hxPlowA7jgArjsMrj2WvjBD/KuxhwWPeCNN7bvQnrmmeRK2C1bksd32w0OOwzOOmv7U1Q9cGfWtWHDkmt9brkFvve95IJPy4/Dohu2bIEVKzoeLfzud21tRo1KwuCMM7Y/RdUTtpl1X6EAs2fD3LlwySV5V9O7OSy6sH59x1B47rm2idn6909OUT311GQgrvVoYcSIfOs2qycnnJCc4XfttQ6LvPX6sNi2DVat6tiN9OqrbW0aGpIwuPTStqOFgw/2KapmWZNgxgz4+tdh6VI45JC8K+q9ev2ps6++mqxVAMkpqgcd1PEU1ZEjPehslpff/z7p3v2TP4F//de8q6k/PnW2TGPGwPXXJwPQhxySDEabWfUYMSKZv2z2bLjyymROM6u8Xj+PqJT0hTY1OSjMqlWhkMxIe8cdeVfSe/X6sDCz6nfSSXDAAclAt+XDYWFmVa9Pn+To4uGHkxmWrfIcFmZWE6ZOTWZVnjUr70p6J4eFmdWEffdNLnb90Y+SFR+tshwWZlYzCoVkBcj580u3tZ7lsDCzmnHqqTB2rAe68+CwMLOa0bcvTJsGCxcmi4NZ5TgszKymTJuWXB/l9bkry2FhZjVlzBg47TS44Ya2ZQAse5mGhaSJklZIWinp8k4ev0rS4vT2vKT17R7fS9JvJX0/yzrNrLYUCrBmDdx3X96V9B6ZhYWkvsDVwGnABGCKpAnFbSLizyPiiIg4AvgP4M52L/Md4OGsajSz2nT66ckEnx7orpwsjyyOAVZGxKqI2ATMASbtoP0U4LbWDUlHAx8CfpFhjWZWg/r1S+Z0u+++ZF17y16WYTEKWF203ZLu60DS/sB44IF0uw/wz8BlGdZnZjVs+vRkPZobbsi7kt4hy7DobAWIrhbPOA+YFxFb0+2vAPdFxOou2idvIM2U1Cypee3atbtQqpnVmgMOgFNOSc6K2rq1dHvbNVmGRQswpmh7NLCmi7bnUdQFBXwcuFTSy8D3gIskXdn+SRFxTUQ0RURTQ0NDz1RtZjWjUIBXXkmuu7BsZRkWTwGNksZLGkASCB0u0pd0EDAMeKx1X0RcEBFjI2Ic8A1gdkR0OJvKzHq3SZOSxZE80J29zMIiIrYAlwILgOXA3IhYKukKSWcWNZ0CzIl6Wd/VzCpm4MBkNtr58+H11/Oupr71+jW4zay2rVgBBx+cLLn6l3+ZdzW1p9w1uH0Ft5nVtIMOgk99Klnnok7+71uVHBZmVvMKBVi5Eh56KO9K6pfDwsxq3llnwdChHujOksPCzGrebrvBhRfCHXfAunV5V1OfHBZmVhcKhWS51dmz866kPjkszKwuHHYYHHts0hXlge6e57Aws7oxcyYsXw6PPpp3JfXHYWFmdePcc2HwYA90Z8FhYWZ1Y4894PzzYe5cWL++dHsrn8PCzOpKoQDvvw+33pp3JfXFYWFmdeXoo+HII+GaazzQ3ZMcFmZWdwoFeOYZ8HRxPcdhYWZ15/zzYffdPdDdkxwWZlZ3hgyBc86B226DDRvyrqY+OCzMrC7NnJkExZw5eVdSHxwWZlaXjjsODjnEXVE9xWFhZnVJSga6n3wSnn0272pqn8PCzOrWhRcmS6/66GLXOSzMrG7tvXey1sXNN8N77+VdTW1zWJhZXSsUkqk/5s3Lu5La5rAws7r26U9DY6O7onZVpmEhaaKkFZJWSrq8k8evkrQ4vT0vaX26/whJj0laKulZSedmWaeZ1a/Wge5f/SqZvtx2TmZhIakvcDVwGjABmCJpQnGbiPjziDgiIo4A/gO4M33oPeCiiDgEmAj8q6ShWdVqZvXt4ouhf3+YNSvvSmpXlkcWxwArI2JVRGwC5gCTdtB+CnAbQEQ8HxEvpPfXAG8ADRnWamZ1bJ99YNIkuPFG+OCDvKupTVmGxShgddF2S7qvA0n7A+OBBzp57BhgAPBiJ4/NlNQsqXnt2rU9UrSZ1adCAdatg7vvzruS2pRlWKiTfV1NGHweMC8itm73AtJI4CbgkojY1uHFIq6JiKaIaGpo8IGHmXXtlFNg3DgPdO+sLMOiBRhTtD0aWNNF2/NIu6BaSdoL+Cnw1xHxeCYVmlmv0acPTJ8O998PL3bop7BSsgyLp4BGSeMlDSAJhPntG0k6CBgGPFa0bwBwFzA7In6cYY1m1otcckkSGh7o7r7MwiIitgCXAguA5cDciFgq6QpJZxY1nQLMidhuTatzgE8BU4tOrT0iq1rNrHcYNQpOPx1uuAE2b867mtqiqJN1B5uamqLZy2KZWQn33gtnnAF33gmTJ+ddTf4kLYqIplLtfAW3mfUqEycmRxge6O4eh4WZ9Sr9+sG0afDzn8Orr+ZdTe1wWJhZrzN9evLn9dfnW0ctcViYWa+z//5w6qlw3XWwdWvp9uawMLNeqlCAlpakO8pKc1iYWa90xhnJnFEe6C6Pw8LMeqUBA2Dq1ORU2tdey7ua6uewMLNea8aMZMzihhvyrqT6OSzMrNdqbISTTkqm/9jWYapSK+awMLNerVCAl16CBzoskGDFHBZm1qtNngx77+2B7lLKCgtJd0g6XZLDxczqyqBBcNFFcNdd4DXUulbul/8PgPOBFyRdKengDGsyM6uoQiGZhfbGG/OupHqVFRYR8V8RcQFwFPAysFDSo5IukdQ/ywLNzLI2YQIcf3wy0F0nE3H3uLK7lSQNB6YCM4CngX8jCY+FmVRmZlZBM2fCihXwyCN5V1Kdyh2zuBN4BNgdOCMizoyI2yPiq8CeWRZoZlYJX/oSDBnige6ulHtk8f2ImBAR/xAR213rWM6iGWZm1W733eGCC2DePHjrrbyrqT7lhsVHJA1t3ZA0TNJXMqrJzCwXhQJs3Ag335x3JdWn3LAoRMT61o2IeAsoZFOSmVk+jjgCmpqSrigPdG+v3LDoI0mtG5L6AgOyKcnMLD+FAixZAk88kXcl1aXcsFgAzJX0GUknA7cBJWeBlzRR0gpJKyVd3snjV0lanN6el7S+6LGLJb2Q3i4u9wcyM9sVU6bAHnt4oLs9RRnHWumV238EfAYQ8AtgVkR0ucZUevTxPPBZoAV4CpgSEcu6aP9V4MiImCZpb6AZaAICWAQcnXZ/daqpqSmam5tL/ixmZqXMmAG33ZZMXb7XXnlXky1Ji8o5Uanci/K2RcQPIuLsiDgrIn64o6BIHQOsjIhVEbEJmANM2kH7KSRHLACfAxZGxJtpQCwEJpZTq5nZrpo5E957LwkMS5R7nUWjpHmSlkla1Xor8bRRwOqi7ZZ0X2evvz8wHmid97Hs55qZ9bSPfQwOP9xdUcXKHbO4gWR+qC3AScBs4KYSz1En+7rq8zoPmFd0tFLWcyXNlNQsqXmtZwAzsx4iJQPdixbB00/nXU11KDcsdouI+0nGOF6JiL8FTi7xnBZgTNH2aGBNF23Po60LquznRsQ1EdEUEU0NDQ0lyjEzK98FFyQz0vroIlFuWGxMB7lfkHSppMnAPiWe8xTQKGm8pAEkgTC/fSNJBwHDgMeKdi8ATk0v/hsGnJruMzOriGHDkilAbrkF3n0372ryV25YfI1kXqj/AxwNfBnY4emsEbEFuJTkS345MDcilkq6QtKZRU2nAHOi6LSsiHgT+A5J4DwFXJHuMzOrmEIB3nkH5s7Nu5L8lTx1Nj0F9sqIuKwyJe0cnzprZj0tIpm+fNgwePTRvKvJRo+dOpsOOh9dfAW3mVlvICXXXDz2GCxdmnc1+Sq3G+pp4B5JF0r6Yusty8LMzKrBxRfDgAEe6C43LPYG1pGcAXVGevvfWRVlZlYtRoyAyZPhppuSGWl7q37lNIqIS7IuxMysWhUKcPvtcOedcP75eVeTj7LCQtINdHJRXERM6/GKzMyqzEknwQEHJF1RvTUsyu2Guhf4aXq7H9gL2JBVUWZm1aRPn2Sg+6GH4Pnn864mH+VOJHhH0e0W4Bzg0GxLMzOrHlOnQt++MGtW3pXko9wji/YagbE9WYiZWTUbORLOOAN+9CPYtCnvaiqv3Fln/yDpndYb8BPgL7MtzcysuhQKsHYtzO8wcVH9K/dsqMFZF2JmVu0+9zkYMyYZ6D777LyrqaxyjywmSxpStD1U0heyK8vMrPr07QvTp8PChfDyy3lXU1nljll8OyLebt2IiPXAt7Mpycysek2blkwDct11eVdSWeWGRWftyurCMjOrJ2PGwMSJcP31sGVL3tVUTrlh0SzpXyR9WNIBkq4CFmVZmJlZtSoUYM0auO++vCupnHLD4qvAJuB2YC7wPvCnWRVlZlbNTj8d9t23d00uWO7ZUO8Cl2dci5lZTejfHy65BL77XWhpgdGj864oe+WeDbVQ0tCi7WGSvMypmfVaM2bAtm1www15V1IZ5XZDjUjPgAIgIt6i9BrcZmZ164AD4JRTkrOitm3Lu5rslRsW2yT9z/QeksbRySy0Zma9SaEAr7ySXHdR78o9/fWvgF9Jejjd/hQwM5uSzMxqw6RJyeJI116bXN1dz8qddfbnQBOwguSMqK+TnBFlZtZrDRyYLLt6zz3w+ut5V5Otcge4Z5CsY/H19HYT8LdlPG+ipBWSVkrq9GwqSedIWiZpqaRbi/b/Y7pvuaR/l6RyajUzq6QZM5KL8370o7wryVa5YxZ/BnwMeCUiTgKOBNbu6AmS+gJXA6cBE4Apkia0a9MIfBM4ISIOAb6W7j8eOAE4nGTdjI8Bny6zVjOzijn4YPjkJ5N1LqKOR3LLDYuNEbERQNLAiPgNcFCJ5xwDrIyIVRGxCZgDTGrXpgBcnZ5dRUS8ke4PYBAwABgI9Afq/CDPzGpVoQArVyYr6dWrcsOiJb3O4m5goaR7gDUlnjMKWF38Gum+YgcCB0r6b0mPS5oIEBGPAQ8Cr6W3BRGxvMxazcwq6uyzYejQ+r6iu9wruCend/9W0oPAEODnJZ7W2RhD+4O0fiSr7p0IjAYekXQoMAL4SLoPkoD6VET8crs3kGaSnpU1dqwX7jOzfOy2G1x4Ifzwh7BuHQwfnndFPa/by6pGxMMRMT/tWtqRFmBM0fZoOh6NtAD3RMTmiHiJ5GyrRmAy8HhEbIiIDcDPgOM6qeWaiGiKiKaGhobu/ihmZj2mUEiWW73pprwrycbOrsFdjqeARknjJQ0AzgPaL0Z4N3ASgKQRJN1Sq4BXgU9L6iepP8ngtruhzKxqHXYYHHts0hVVjwPdmYVFRGwBLgUWkHzRz42IpZKukHRm2mwBsE7SMpIxissiYh0wD3gRWAI8AzwTET/JqlYzs55QKMCyZfDoo3lX0vMUdRKBTU1N0dzcnHcZZtaLbdgAI0fCWWfVznUXkhZFRFOpdll2Q5mZ9Sp77gnnnw9z58L69aXb1xKHhZlZDyoU4P334dZbS7etJQ4LM7MedPTRcOSR9TfQ7bAwM+tBUnJ0sXgxLFqUdzU9x2FhZtbDzj8fdt+9vq7odliYmfWwIUPgnHOScYsNG/Kupmc4LMzMMlAoJEExZ07elfQMh4WZWQY+/nGYMKF+uqIcFmZmGWgd6H7ySXj22byr2XUOCzOzjFx4YbL0aj0cXTgszMwyMnx4MvXHzTcnF+rVMoeFmVmGCoVk6o958/KuZNc4LMzMMvTpT0NjY+13RTkszMwyJMGMGfDII/Cb3+Rdzc5zWJiZZezii6Ffv9o+unBYmJll7EMfgkmT4MYb4YMP8q5m5zgszMwqoFCAdevg7rvzrmTnOCzMzCrgs5+F/fev3a4oh4WZWQX06ZMMdN9/P7z4Yt7VdJ/DwsysQi65JAmN667Lu5Luc1iYmVXIqFFw+ulwww2weXPe1XRPpmEhaaKkFZJWSrq8izbnSFomaamkW4v2j5X0C0nL08fHZVmrmVklFArwu9/BvffmXUn3ZBYWkvoCVwOnAROAKZImtGvTCHwTOCEiDgG+VvTwbOCfIuIjwDHAG1nVamZWKaedBvvtV3sD3VkeWRwDrIyIVRGxCZgDTGrXpgBcHRFvAUTEGwBpqPSLiIXp/g0R8V6GtZqZVUS/fjBtGvz85/Dqq3lXU74sw2IUsLpouyXdV+xA4EBJ/y3pcUkTi/avl3SnpKcl/VN6pLIdSTMlNUtqXrt2bSY/hJlZT5s+Pfnz+uvzraM7sgwLdbIv2m33AxqBE4EpwCxJQ9P9nwS+AXwMOACY2uHFIq6JiKaIaGpoaOi5ys3MMjRuHJx6ahIWW7fmXU15sgyLFmBM0fZoYE0nbe6JiM0R8RKwgiQ8WoCn0y6sLcDdwFEZ1mpmVlGFAqxeDQsW5F1JebIMi6eARknjJQ0AzgPmt2tzN3ASgKQRJN1Pq9LnDpPUerhwMrAsw1rNzCrqjDNgn31qZ6A7s7BIjwguBRYAy4G5EbFU0hWSzkybLQDWSVoGPAhcFhHrImIrSRfU/ZKWkHRp1chHamZW2oABMHUq/OQn8NpreVdTmiLaDyPUpqampmhubs67DDOzsr3wAhx4IPz938O3vpVPDZIWRURTqXa+gtvMLCeNjXDiiTBrFmzblnc1O+awMDPLUaEAL70EDzyQdyU75rAwM8vRF78Ie+9d/QPdDgszsxwNGgQXXQR33QXVfG2xw8LMLGeFQjIL7ezZeVfSNYeFmVnOJkyA449PuqKq9QRVh4WZWRUoFGDFCnjkkbwr6ZzDwsysCnzpS7DXXtU70O2wMDOrAnvsARdcAPPmwVtv5V1NRw4LM7MqUSjAxo1w8815V9KRw8LMrEoceSQ0NVXnQLfDwsysihQKsGQJPPlk3pVsz2FhZlZFpkxJxi+qbaDbYWFmVkUGD4bzzoM5c+Cdd/Kupo3DwsysyhQK8O67cNtteVfSxmFhZlZljjkGDjusurqiHBZmZlVGSo4uFi2Cp5/Ou5qEw8LMrAp9+cvJjLTVcnThsDAzq0LDhiVTgNxySzJ+kTeHhZlZlSoUkjOifvzjvCvJOCwkTZS0QtJKSZd30eYcScskLZV0a7vH9pL0W0nfz7JOM7Nq9IlPwMEHV0dXVGZhIakvcDVwGjABmCJpQrs2jcA3gRMi4hDga+1e5jvAw1nVaGZWzSSYMQMefRSWLs23liyPLI4BVkbEqojYBMwBJrVrUwCujoi3ACLijdYHJB0NfAj4RYY1mplVtYsugv798z+6yDIsRgGri7Zb0n3FDgQOlPTfkh6XNBFAUh/gn4HLMqzPzKzqNTTA5Mlw003JjLR5yTIs1Mm+9vMo9gMagROBKcAsSUOBrwD3RcRqdkDSTEnNkprXVvNK52Zmu6BQgDffhDvvzK+GLMOiBRhTtD0aWNNJm3siYnNEvASsIAmPjwOXSnoZ+B5wkaQr279BRFwTEU0R0dTQ0JDFz2BmlruTT4bx4/PtisoyLJ4CGiWNlzQAOA+Y367N3cBJAJJGkHRLrYqICyJibESMA74BzI6ITs+mMjOrd336JEcXDz0EL7yQUw1ZvXBEbAEuBRYAy4G5EbFU0hWSzkybLQDWSVoGPAhcFhHrsqrJzKxWTZ0KffvCrFn5vL+i2pZj2klNTU3R3NycdxlmZpmZPDk5jXb1ahgwoGdeU9KiiGgq1c5XcJuZ1YhCAd54A+a379CvAIeFmVmN+NznYMyYfAa6HRZmZjWib1+YNg0WLoSXX67sezsszMxqyLRpyZ/XXVfZ93VYmJnVkLFj4bTT4PrrYcuWyr2vw8LMrMYUCrBmDfzsZ5V7T4eFmVmNOf102Hffyg50OyzMzGpM//5wySXw05/Cb39bmfd0WJiZ1aDp02HbtmTsohIcFmZmNejDH4bPfCY5K2rbtuzfz2FhZlajCgV45ZXkuousOSzMzGrUF74Aw4dXZqC7X/ZvYWZmWRg4EP7iL+C99yAiWbM7Kw4LM7Ma9q1vVeZ93A1lZmYlOSzMzKwkh4WZmZXksDAzs5IcFmZmVpLDwszMSnJYmJlZSQ4LMzMrSRGRdw09QtJa4JVdeIkRwO97qJye5Lq6x3V1j+vqnnqsa/+IaCjVqG7CYldJao6IprzraM91dY/r6h7X1T29uS53Q5mZWUkOCzMzK8lh0eaavAvoguvqHtfVPa6re3ptXR6zMDOzknxkYWZmJfXasJD0JUlLJW2T1OVZBJImSlohaaWkyytQ196SFkp6If1zWBfttkpanN7mZ1jPDn9+SQMl3Z4+/oSkcVnV0o2apkpaW/T5zMi6pvR9r5f0hqTnunhckv49rftZSUdVSV0nSnq76PP6mwrVNUbSg5KWp7+Lf9ZJm4p/ZmXWVfHPTNIgSU9Keiat6+86aZPd72NE9Mob8BHgIOAhoKmLNn2BF4EDgAHAM8CEjOv6R+Dy9P7lwHe7aLehAp9RyZ8f+Arw/9L75wG3V0FNU4Hv5/Bv6lPAUcBzXTz+eeBngIDjgCeqpK4TgXtz+LxGAkel9wcDz3fyd1nxz6zMuir+maWfwZ7p/f7AE8Bx7dpk9vvYa48sImJ5RKwo0ewYYGVErIqITcAcYFLGpU0Cbkzv3wh8IeP325Fyfv7ieucBn5GyXNwxl7+TskTEL4E3d9BkEjA7Eo8DQyWNrIK6chERr0XEr9P7fwCWA6PaNav4Z1ZmXRWXfgYb0s3+6a39oHNmv4+9NizKNApYXbTdQvb/aD4UEa9B8o8W2KeLdoMkNUt6XFJWgVLOz/8/bSJiC/A2MDyjesqtCeCstNtinqQxGdbTHXn8eyrXx9PujZ9JOqTSb552lxxJ8r/lYrl+ZjuoC3L4zCT1lbQYeANYGBFdfl49/ftY12twS/ovYN9OHvqriLinnJfoZN8unz62o7q68TJjI2KNpAOAByQtiYgXd7W2dsr5+TP5jHagnPf7CXBbRHwg6Y9J/qd1coY1lavSn1W5fk0y5cMGSZ8H7gYaK/XmkvYE7gC+FhHvtH+4k6dU5DMrUVcun1lEbAWOkDQUuEvSoRFRPBaV2edV12EREafs4ku0AMX/Kx0NrNlwURgKAAADyUlEQVTF19xhXZJelzQyIl5LD7ff6OI11qR/rpL0EMn/fno6LMr5+VvbtEjqBwwh2y6PkjVFxLqizWuB72ZYT3dk8u9pVxV/EUbEfZL+U9KIiMh8DiRJ/Um+kG+JiDs7aZLLZ1aqrjw/s/Q916e/9xOB4rDI7PfR3VA79hTQKGm8pAEkA0aZnXmUmg9cnN6/GOhwBCRpmKSB6f0RwAnAsgxqKefnL673bOCBSEfXMlKypnZ92meS9DlXg/nARekZPscBb7d2OeZJ0r6t/dqSjiH5Xli342f1yPsKuA5YHhH/0kWzin9m5dSVx2cmqSE9okDSbsApwG/aNcvu97GSo/nVdAMmk6TwB8DrwIJ0/37AfUXtPk9yNsSLJN1XWdc1HLgfeCH9c+90fxMwK71/PLCE5EygJcD0DOvp8PMDVwBnpvcHAT8GVgJPAgdU4DMqVdM/AEvTz+dB4OAK/Zu6DXgN2Jz+25oO/DHwx+njAq5O615CF2fh5VDXpUWf1+PA8RWq6xMkXSTPAovT2+fz/szKrKvinxlwOPB0WtdzwN+k+yvy++gruM3MrCR3Q5mZWUkOCzMzK8lhYWZmJTkszMysJIeFmZmV5LAw6wZJG0q32uHz56VX3SNpT0k/lPRiOovoLyUdK2lAer+uL5q12uKwMKuQdP6gvhGxKt01i+Tq2saIOIRkttwRkUyQeD9wbi6FmnXCYWG2E9Iriv9J0nOSlkg6N93fJ536YamkeyXdJ+ns9GkXkF6RL+nDwLHAX0fENkimbomIn6Zt707bm1UFH+aa7ZwvAkcAHwVGAE9J+iXJ1CvjgMNIZgxeDlyfPucEkqupAQ4BFkcyMVxnngM+lknlZjvBRxZmO+cTJDPbbo2I14GHSb7cPwH8OCK2RcTvSKYbaTUSWFvOi6chsknS4B6u22ynOCzMdk5XC8rsaKGZ90nm7oFkXqGPStrR7+BAYONO1GbW4xwWZjvnl8C56WI0DSRLlz4J/Ipk4aU+kj5Esvxmq+XA/wKIZO2RZuDvimYvbZQ0Kb0/HFgbEZsr9QOZ7YjDwmzn3EUy++czwAPA/027ne4gmdn1OeCHJCusvZ0+56dsHx4zSBbBWilpCcnaG61rNZwE3Jftj2BWPs86a9bDJO0ZyQpqw0mONk6IiN+laxA8mG53NbDd+hp3At+M0uvEm1WEz4Yy63n3povUDAC+kx5xEBHvS/o2yTrJr3b15HRRp7sdFFZNfGRhZmYleczCzMxKcliYmVlJDgszMyvJYWFmZiU5LMzMrCSHhZmZlfT/AbmjZnEnLMPBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "C_s = np.logspace(-1, 3, 5)#logspace(a, b, N)把10的a次方到10的b次方分成N份\n",
    "\n",
    "accuracy_s = []\n",
    "for i, oneC in enumerate(C_s):\n",
    "    tmp = fit_grid_point_Linear(oneC, x_train_part, y_train_part, x_val, y_val)\n",
    "    accuracy_s.append(tmp)#将超参数校验集上的性能存起来，每个超参数在校验集上的一个分数，然后就可以求得最佳的超参数\n",
    "    \n",
    "    \n",
    "x_axis = np.log10(C_s)\n",
    "#输出不同超参数对应模型的正确率的变化\n",
    "plt.plot(x_axis, np.array(accuracy_s), 'b-')\n",
    "\n",
    "plt.legend()\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('accuracy')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "code_folding": [],
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10.0\n"
     ]
    }
   ],
   "source": [
    "#最佳超参数\n",
    "index = np.argmax(accuracy_s, axis=None)\n",
    "Best_C = C_s[index]\n",
    "print(Best_C)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 根据6中得到的训练数据，训练RBF核SVM，并对超参数（C和gamma）进行超参数调优，并分析参数C和gamma对模型复杂度的影响，以及对训练误差和交叉验证得到的测试误差的影响。（20分） "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit_grid_point_RBF(C, gamma, x_train, y_train, x_val, y_val):\n",
    "    \n",
    "    # 在训练集是那个利用SVC训练\n",
    "    SVC3 =  SVC( C = C, kernel='rbf', gamma = gamma)\n",
    "    SVC3 = SVC3.fit(x_train, y_train)\n",
    "    \n",
    "    # 在校验集上返回accuracy\n",
    "    accuracy = SVC3.score(x_val, y_val)\n",
    "    \n",
    "    print(\"C= {} and gamma = {}: accuracy= {} \" .format(C, gamma, accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#对超参数（C和gamma）进行超参数调优\n",
    "accuracy_s = np.matrix(np.zeros(shape=(5, 3)), float)\n",
    "gamma_s = np.logspace(-1, 1, 3)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 0.1 and gamma = 0.1: accuracy= 0.7108022668568564 \n",
      "C= 0.1 and gamma = 1.0: accuracy= 0.7350321909094414 \n",
      "C= 0.1 and gamma = 10.0: accuracy= 0.3282316203400285 \n"
     ]
    }
   ],
   "source": [
    "oneC = 0.1\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    accuracy_s[0,j] = fit_grid_point_RBF(oneC, gamma, x_train_part, y_train_part, x_val, y_val)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 1 and gamma = 0.1: accuracy= 0.7491614942750299 \n",
      "C= 1 and gamma = 1.0: accuracy= 0.7792898723929218 \n",
      "C= 1 and gamma = 10.0: accuracy= 0.5959944485138209 \n"
     ]
    }
   ],
   "source": [
    "oneC = 1\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    accuracy_s[1,j] = fit_grid_point_RBF(oneC, gamma, x_train_part, y_train_part, x_val, y_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 10 and gamma = 0.1: accuracy= 0.7726974825552257 \n",
      "C= 10 and gamma = 1.0: accuracy= 0.79181926828328 \n",
      "C= 10 and gamma = 10.0: accuracy= 0.6120320752534794 \n"
     ]
    }
   ],
   "source": [
    "oneC = 10\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    accuracy_s[2,j] = fit_grid_point_RBF(oneC, gamma, x_train_part, y_train_part, x_val, y_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 100 and gamma = 0.1: accuracy= 0.7842245267743552 \n",
      "C= 100 and gamma = 1.0: accuracy= 0.7804849840009253 \n",
      "C= 100 and gamma = 10.0: accuracy= 0.6113766914684452 \n"
     ]
    }
   ],
   "source": [
    "oneC = 100\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    accuracy_s[3,j] = fit_grid_point_RBF(oneC, gamma, x_train_part, y_train_part, x_val, y_val)   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 1000 and gamma = 0.1: accuracy= 0.7726974825552257 \n",
      "C= 1000 and gamma = 1.0: accuracy= 0.7750106017965226 \n",
      "C= 1000 and gamma = 10.0: accuracy= 0.6113766914684452 \n"
     ]
    }
   ],
   "source": [
    "oneC = 1000\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    accuracy_s[4,j] = fit_grid_point_RBF(oneC, gamma, x_train_part, y_train_part, x_val, y_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNX9+P/XO/u+kQWSEMISWRIhQFgErbijomg/tGhr3bePy8el9au2tP0UsWJbbP1Va7WWutSKrSv1Y2txq20VJCAii8oiSwgYICEJCVnn/P64N5PJxkyWm5vl/Xw85pGZO+fOvGeSzHvOPee8rxhjUEoppY4nyO0AlFJK9X2aLJRSSvmlyUIppZRfmiyUUkr5pclCKaWUX5oslFJK+aXJQimllF+aLJRSSvmlyUIppZRfIW4H0FOSk5NNdna222EopVS/sm7dukPGmBR/7QZMssjOzqawsNDtMJRSql8Rkd2BtNPDUEoppfzSZKGUUsovTRZKKaX80mShlFLKL00WSiml/NJkoZRSyi9NFkoppfwaMOsslOpTGuuhbDcc+gIObwMEYtIgJtX+mQaRiRCk39dU/6DJQqnuOHYEDm+3ksKhL+DQNutn6U7wNBx/36AQiE6FmJS2icT3enQKhMeCSO+8JqXaoclCKX88Hijfa/UQmpJB08+jXzW3CwqBpNGQfAKMm2f9TD4BkscAAkdLrPZHv/K57rPtwKfWbdPYNoaQSJ8E0jqp+GyPToXQiF57a9TgoclCqSZ11T69BJ+kcHg7NBxrbheRYCWBMWdBck5zUkgcAcGhHT9+RJydOI7D44FjpR0kFfv64R2w+wOrXbvPE98ymUSntt9riU6GoODOv09qUNJkoQYXY6wPXO9hI5/kUL7Hp6FYH/7JJ8CoU1smhaghzh0SCgqyPsSjkyEt9/htG+qg6qD1epp+tk4wxR9bP+uOtt1fgiAquW3vpE1SSbHGV/Qw2KCmyUINTA11UPZl27GEQ9ugtqK5XWi09W0/awYkf6c5KSSN7vuHc0LCID7DuvhTV9Wyd9I6qVSVWO/N0a+gsa7t/sFhrXooqe0fBotJg7Donn+tynWaLFT/Vl3qkwi+aD6MVPply2P/selWIpi40O4h2EkhLn1wfGMOi4akkdbleIyBmiPtjKn4JJryIti3zurNYNp5rpi24yhtEozdYwkJc+Tlqp6nyUL1fZ5GOLKnZVJoul59qLldcBgMGQOpE2DCRT5JIceaTaT8E7EOOUUmQsrY47dtbIDqw/ZhsJL2E0zJVjj6HtSUt/8YkUmteid2MonLgIQs6xKdqlOM+wBNFqrvqD3aasZR0wDzDmisbW4XNcSecXSez4yjHEgYoQO2vSk4BGLTrIs/9TX2uEp7h8HsMZeitZjKrxDfyQSAJyiMmugM6xKTSU20fYnJoCY6k/qI5MHROzyO2IgQpmQlOvocmixU7zIGKorbTwoV+5rbSRAkjrRnHZ3RnBSG5ED0EPfiV10TGgEJw62LraKmni3FFWzaV86W4go2V1SwvbqSSE81Q6WUTDlIphwiUw4yvL6EzPJiMuUT0qWyxUPXmFD2mWSKTIrPJZm9JpUik8Ih4oCBnUzyhyfw6s2zHX0OR5OFiMwFHgaCgSeNMUtb3Z8FPA0k2G3uMca8Yd93L3AN0Aj8jzHmTSdjVT2svsZamNZicNkeU/CdmRMWa/UKsk9pOeMoaSSEhLsXv+pRBytr2VRsJ4Xicjbtq2BPabX3/tTYcHLT4zg7N43c9DhSYttOLmgAdgF76qsIO7qPsKNFhB8tIuxoEUlHixhauZfZR9cTUlvWYj9PcDi1MZnU2Zem67Wx1s+GAdAziQ53vkftWLIQkWDgUeAsoAhYKyIrjTFbfJotAv5sjHlMRCYAbwDZ9vVLgFwgHXhLRE4wpr3VSso1xljHrFvPNjr0BRzZDcbT3DZ+uJUMsi6zxhWakkLs0H7/j6qaGWMoKjvG5uJyNhdXsNnuOZRUNh9GzEqKIi8jjoXThjMhPY7c9DhS20kOHUsEMoEZ7d9dWwlH9lrjXEf2EHRkN5FH9hB5ZA/s+bTt+pSQyObxkRaXEfaYSf9PJj3ByZ7FdGC7MWYngIisAOYDvsnCAHH29Xig2L4+H1hhjKkFvhSR7fbjfehgvMofjwf2roZNL8OBjVZSOObzLS4kwkoE6fkw8ZvNYwlDxuh0ygGo0WPYefConRSs3sKW/RWUH6sHIEggJzWWk8ckMyE9jryMeCakxxEXcZyFiz0hPBbSJliX9tRWehNJ82W39XNfYcu/aWhOJokj2k8oTq676UOcTBYZwF6f20W0/Srwv8A/RORWIBo402ff1a32DWAyuXLEwc9h4wuw8S/WwrXQKEif0nbGUfxwHWAeoGobGvniwFErKdi9hs/2V3Ks3ursh4UEMX5oLOedOIxcOzGMGxpLRGgf/HsIj7UWPHa06LGmwirvcmSPVQzSN5ns/ciaWuwrNKptb6RFMkkaEMnEyWTR3rvTelL2pcBTxphlInIS8KyI5AW4LyJyPXA9QFZWVjfDVS1UHoBNL1lJYv8n1oDzqNPg9EUw7nwIj3E7QuWQo7UNbN1vHT5qOpS07atKGjzWv2BMeAgT0uO4ZPpw8tLjyc2IY3RKDKHBA2R6a0QcRBwvmZS3OMzVnEx2w941bacJh0a37ZF4eykj+s3qeCeTRREw3Od2Js2HmZpcA8wFMMZ8KCIRQHKA+2KMeQJ4AqCgoKCd1UGqU2qPwmevWwli53vWmMOwfDjnAcj7r8CmSKp+pbSqznsIabM9AP3l4SqM/d80JDqM3Ix45oxNsRJDehxZSVEEBfX9DzfHRMTD0HgYmtf+/ceONPdMWieUPauhtlUyCYvpYMykbyUTJ5PFWiBHREYC+7AGrL/Vqs0e4AzgKREZD0QAB4GVwJ9E5CGsAe4c4CMHYx28GhusxLDxBStR1FdDfBacfKc17uBvYZbqF4wx7C+vadFb2Fxczv7yGm+bjIRIctPjuGhyBrnpceSmx5MWF470gQ+qfiUywboMPbH9+48daSeR2JfdH7QsRwPWjMEOk0lWryUTx5KFMaZBRG4B3sSaFrvcGLNZRBYDhcaYlcB3gd+JyB1Yh5muNMYYYLOI/BlrMLwBuFlnQvUgY6wCcxv/DJtetBZERSRYpTAmLoThM3TFbD/m8Ri+PFzlTQib7V5DWbU18CwCo5KjmT4yyZsUctPjSIjS0hu9oimZDJvY/v3HjjSPkbS+7Po31LVcZ0J4HIz8GlzynKNhizED4+hNQUGBKSwsdDuMvq1slzVIvfEFa1FccBicMNdKEDln6bqGfqiuwcO2kkorMdi9hq37K6iqs75bhQYLJ6TFescWctPjGT8slqgwXY/bLzXV7vJNIGW7rUH0Ofd06SFFZJ0xpsBfO/2LGeiqS2HzK1YvYq89wWzEbJh1C0yYb3VhVb9QXdfA1v2VbGkaY9hfzhcHjlLXaK1niQoLZsKwOBZMzbR6Cxlx5KTGEhaivcQBw7d217BJvfrUmiwGovoa2PYmfPICbPsHeOoheSyc8SM48RvWcU7Vp5VX17eYprq5uIKdB49iT0giMSqU3PR4rpqdTW6GdRgpe0g0wYN54Fk5SpPFQOHxwJ4PrENMm1+zZlzEpMGMG6yB6qET+8SMCtWSMYaSylqfgWer17DvSHMxvWHxEeSmx3G+vYYhNyOe9PgIHXhWvUqTRX9XsrV5wVxFkTWne/wFMGkhjDxVF8n1IR6PYU9pdXNSKK5gS3E5h442n2xoZHI0+VkJXDZzhD34HMeQGB1LUu7TZNEfVey3ZjFtfAEOfAoSDKNPhzP/1yrbraU1+oTDR2tZu6uMwl2lbNxXztbiCiprGwAICRLGpMYwZ2yqd0bS+GGxxDpdCkOpLtJk0V/UVsJWe8Hcl/+0FsylT4G5D0Le160TxijXGGP1GtbuKmPtl6Ws3V3KzoNVgFUKY8KwOOZPTic3PZ689Hhy0mL6ZikMpTqgyaIva6yHHe/aC+b+DxqOWSs6T/meXagvx+0IB61Gj2Hr/grW7iqlcFcZa3eVeiurxkWEMC07iW9MHc607EROzIwnPEQTg+rfNFn0NcbAvvVWgtj0knXa0IgEyL8UJl4Cw6frQLULauob+XjPEQp3lfLRrlI+3nOEo/YhpYyESE4aPYSC7CSmZyeRkxozuMthqAFJk0VfUbqzecFc6Q4IDoex9oK5MWfpie17WVlVHYW7y7zJYdO+cuobrXmrY9NiuWhyOtOykyjITiIjIdLlaJVyniYLN1WX2pVd/wxFdumr7FPg5Nth/IVWSQDluKYT9qzdVeodkN5WYp3NLyw4iImZ8Vxz8iimZScydUSilsVQg5Imi95Wfwy++LuVILb9AzwNkDLemsmUt6DFOYqVMxo9hs8PVFK4u5SPvrTGHA5UWAX1YsNDmJqdyEWTM5iWncTEzHgdiFYKTRa9w+OB3f+2DjFtWWlVlYwdBjP/2zrMlJan4xAOqqlvZGNRud1zKGXd7jIqa6zxhqFxEUwbmcS07EQKRiQxdmisroJWqh2aLJz01WYrQXz6IlTss+rWj7/Qmsk08mu6YM4h5dX1rNtTykdf2mscisq99ZNyUmOYNzGdadmJTMtOIjMxUldCKxUATRY9raIYPv2LdZjpq03WgrkxZ8JZi2HseRAW5XaEA07xkWPeXsPaL8v4/CurhHNIkHBiZjxXzs5mWnYSU0ckkhSt4w1KdYUmi55QUwFb/2ovmHsfMJBRAOf+HHIvhpgUtyMcMDwew7aSo97kULirzFtHKSY8hCkjEpk3cRgF2UnkD08gMkx7b0r1BE0WXdVYD9vfthLE529AQw0kjoRT77YOMw0Z7XaEA0JtQyOb9pV7DykV7i6j/Jh1Ep+U2HCmZydx7SkjmZadxLihsYQMlPNAK9XHaLLoDGOgqNCu7PoyVB+GyCSYfJk1UJ05TQequ6mipp519vqGtV+WsaHoCHUN1njDqJRo5uYO9Q5IZyVF6XiDUr1Ek0UgDu+wxyFesBbPhUTA2HOtBDH6DF0w1w0Hymuaxxt2lfHZgQqMgeAgIS8jnstnjqAgO4mC7ESStfqqUq7RZNGRqkP2GeZegKK1gMDIU+CU71olwCPi3Y6w3zHGsOPgUe8hpbW7S9lbao03RIUFMyUrkdvOyGF6dhL5WQl66k+l+hD9b/RVf8waf9j4Z9j+lrVgLjXXmsmUtwDiM9yOsF+pa/CwqbjcSgz2yuiyamu8ITkmjIIRSVw5ayTTshOZMCxOxxuU6sM0WXgaYde/rASxZSXUVUJsOsy8yTrMNDTP7Qj7jaO1Daz3qae0Ye8Rauqt8YaRydGcOT6NadlJTBuZRPYQHW9Qqj/RZFG+F56ZD2GxMGG+NZMp+2RdMBeAkooa6/wNu0op3F3KluIKPAaCBHLT47l0ehbTs5OYmp1IamyE2+EqpbpBk0ViNlz2MoyYBaFaPbQjHo813mBVYi2jcHcpuw9XAxARGsTk4YnccnoO07ITmZyVSEy4/mkpNZDofzTAmDPcjqDPqa5r4JO95azbbdVSWr/niHd9Q1J0GAUjErlsxgimjUwiNz2OUB1vUGpA02ShAGsKa6GdGNbtLmNLcQUNHuv8DTmpMZybN5SpIxIpyNbxBqUGI00Wg1BDo4fPDlR6E8O63c0lMyJCg8gfnsANp46iYEQSk7MS9PwNSilNFoNBRU09H+85YieGUjbsOUJVXSMAaXHhFIxI4pqTRzJ1RCIT9JCSUqodmiwGGGMMe0uPsW6PVWRv3W6rCquxZymNGxrHf03NZOoI66xvGQlaolsp5Z+jyUJE5gIPA8HAk8aYpa3u/yVwmn0zCkg1xiTY9zUCn9r37THGXOhkrP1VXYOHzcXldj2lMtbtKeNgZS1gVWGdnJXAuXnDmDoikfysBJ2lpJTqEsc+OUQkGHgUOAsoAtaKyEpjzJamNsaYO3za3wpM9nmIY8aYfKfi669Kq+qshW+7y1i/u4xPio5QaxfaG54UycljkpkyIpGCEYmckKZnfVNK9Qwnv2ZOB7YbY3YCiMgKYD6wpYP2lwI/djCefseqpVTlnb5auLuMnQerAAgNFnLT47ls5ggK7ENKqXG68E0p5Qwnk0UGsNfndhEwo72GIjICGAm847M5QkQKgQZgqTHmVacC7Stq6hv5ZO8R1u0pY519SOmIXUspMSqUqSMSWTA1k4IRSUzMjCciVFeZK6V6h5PJor3jH6aDtpcALxpjGn22ZRljikVkFPCOiHxqjNnR4glErgeuB8jKyuqJmHtVSUUNhfbU1cLdZWzeV+5d2zA6JZqzJ6RRMCKJKSMSGZ0SrQPRSinXOJksioDhPrczgeIO2l4C3Oy7wRhTbP/cKSLvYY1n7GjV5gngCYCCgoKOElGf0OgxfH6g0u41lLJuT5m3PHd4SBCTMhO47mujKBhhlcvQc0UrpfoSJ5PFWiBHREYC+7ASwrdaNxKRsUAi8KHPtkSg2hhTKyLJwGzgZw7G2uOO1jbw8Z7mRW8f7znC0doGwDodaMGIRK44KZupIxLJTY8nLETXNiil+i7HkoUxpkFEbgHexJo6u9wYs1lEFgOFxpiVdtNLgRXGGN+ewXjgcRHxAEFYYxYdDYy7zhjDviPHmqev7rbO+OYx1llWx6bFctHkdKtcxogkMhN1bYNSqn+Rlp/R/VdBQYEpLCzsleeqb/SwpbjCO321cHcpX1VYaxuiw4KZnJXoXfQ2OSuB2IjQXolLKaU6S0TWGWMK/LXTFVoBOFJdx3r7kFLhLmttQ9NJfTISIpk5aog3OYxNi9UzvimlBhxNFq0YY/jyUJVPr6GM7SVHAQgJEnLT47h0ehYFI5KYOiKRofG6tkEpNfAN+mRR1+Dhk6Ij3l7D+j1llFbVARAfGcqUrAQunpzB1BGJTMpMIDJM1zYopQafQZ8sSqvq+MZvrYlYo5KjOX1cqndF9OiUGIK0XIZSSmmyGBofwR+umsbEjHiGxIS7HY5SSvVJgz5ZAJw2NtXtEJRSqk/TaTtKKaX80mShlFLKL00WSiml/NJkoZRSyi8d4FbKYdX11dR76gkNCiUkKISQoBCCRL+nqf5Fk4VSXVTvqefwscOUVJdQUl3CV9VfcbD6oPd2yTHrZ1V9VZt9gySoRfIIEeun77am6y3atb4t7bdtvU+7jyOhhAaHeh/D336tf2rCG1w0WSjVijGGirqK5g/91hc7CRw+dhjT6nxeIUEhpESmkBqVypiEMcxKn0VKZArhweE0eBqo99Q3/zQN1vXG5uu+bXwv9Z56ahpqvPvVN9ptOtivscV5xJwRJEHNyaqDpNNe8muR5NrZLzQolOCgYAQhSIIIkiBEhGAJtq77bPfeH+g2EYII6tH9m7YFS7B13Wf/jl5HR9v6cjVqTRZqUKltrKWkusTbA/DtDXxV/RUHj1nXaxtr2+ybEJ5AalQqqVGpjE8aT0qUlRTSotK8CSIxIrFPfOP2GE+7SaRFIjINbRPYcZKV7371jfUtEl57+7V32zfhtZcom9o1mkaMMXiMp01CHsiakkh7yfF428Ynjefh0x92NDZNFmpA8BgPpTWl3kTwVfVX1vVjPterD3Kk9kibfcODw71JIC85j9RI63pqdKr3ekqU1TvoL4IkiLDgMMKC+/8ZF5uShgdP8/WmS3vb7ATTeltX9m+dtFpva9q/xTa7bbvbPI0tYmva1t3Xlhmb6fjvQZOF6vOq66u9PQDfJODbMzh47CANnoYW+wlCcmQyKVEpZMRkMCV1ircHkBaV5u0ZxIXF9enu/2DX9O05GC3i6SZNFso1DZ4GDh075Lc3cLT+aJt9Y0JjvN/4pw2dZl2PTCEtKs27PTkymZAg/RNXqifof5LqcccdID7WfL3dAWIJISUqhZSolBYDxL69gbSoNKJCo1x6dUoNTposVKeV15az48gO64O/qm0SOFh9kJrGmjb7NQ0Qp0SlMC5pXLu9gaSIpD4xQKyUakmTheqUA1UH+K+V/0VFXYV3W9MAcUpkCnlD8kgdnuodMG669LcBYqVUS5osVKcs/WgpdY11PHzawwyPHa4DxEoNEposVMDe2/seb+95m9un3M7pWae7HY5SqhfpwWEVkOr6ah5Y8wBjEsZwee7lboejlOpl2rNQAXl84+MUVxXz1NynCA0KdTscpVQvC6hnISIvicj5IjpNZTDaVraNZzY/w8VjLmZq2lS3w1FKuSDQD//HgG8B20RkqYiMczAm1Yd4jIclq5cQExbDHVPvcDscpZRLAkoWxpi3jDHfBqYAu4BVIvKBiFwlInpMYgB7bftrrC9Zz51T7yQxItHtcJRSLgn4sJKIDAGuBK4FPgYexkoeqxyJTLmurKaMZeuWMSV1CvPHzHc7HKWUiwIds3gZ+BcQBVxgjLnQGPOCMeZWIOY4+80Vkc9FZLuI3NPO/b8UkQ325QsROeJz3xUiss2+XNH5l6a665frfklVXRWLZi7SVdVKDXKBzoZ6xBjzTnt3GGMK2tsuIsHAo8BZQBGwVkRWGmO2+Ox7h0/7W4HJ9vUk4MdAAWCAdfa+ZQHGq7pp3VfreGX7K1yddzU5iTluh6OUclmgXxfHi0hC0w0RSRSRm/zsMx3YbozZaYypA1YAxzuWcSnwvH39HGCVMabUThCrgLkBxqq6qb6xniWrl5Aenc4NE29wOxylVB8QaLK4zhjjPURkf4Bf52efDGCvz+0ie1sbIjICGAk09V4C3lf1vGe2PMP2I9v5/ozva3VXpRQQeLIIEp/iP/YhJn+n4GqvWFBH50e8BHjRGO+JgwPaV0SuF5FCESk8ePCgn3BUIPYd3cdvP/ktZ2SdwanDT3U7HKVUHxFosngT+LOInCEip2MdLvq7n32KgOE+tzOB4g7aXkLzIaiA9zXGPGGMKTDGFKSkpPgJR/ljjOGna36KiHDP9DbzEZRSg1igyeJurENE/w3cDLwN/D8/+6wFckRkpIiEYSWEla0bichYIBH40Gfzm8DZ9thIInC2vU056J097/B+0fvcnH8zQ6OHuh2OUqoPCWg2lDHGg7WK+7FAH9gY0yAit2B9yAcDy40xm0VkMVBojGlKHJcCK4wxxmffUhG5DyvhACw2xpQG+tyq86rqq3jgowcYmziWb4//ttvhKKX6mICShYjkAA8AE4CIpu3GmFHH288Y8wbwRqttP2p1+3872Hc5sDyQ+FT3/WbDbyipLmHZnGV63mqlVBuBHob6A1avogE4DXgGeNapoFTv+qz0M57b+hwLTljApJRJboejlOqDAk0WkcaYtwExxuy2ewN69psBwGM83PfhfcSHx3PblNvcDkcp1UcFeryhxi5Pvs0eh9gHpDoXluotL37xIhsPbeSnJ/+U+PB4t8NRSvVRgfYsbseqC/U/wFTgMkDrNfVzh44d4lfrf8X0odOZN2qe2+Eopfowvz0LewHeN40xdwFHgascj0r1imWFy6hpqGHRzEX4rLlUSqk2/PYs7FXVU0U/TQaUNfvX8PrO17k672pGxo90OxylVB8X6JjFx8BrIvIXoKppozHmZUeiUo6qa6xjyeolDI8dzrUnXut2OEqpfiDQZJEEHKblDCgDaLLoh5ZvWs6uil389szfEhES4X8HpdSgF+gKbh2nGCD2VOzhdxt/x9zsuczOmO12OEqpfiLQFdx/oJ2qr8aYq3s8IuUYYwz3r7mfsOAw7pp2l9vhKKX6kUAPQ73ucz0CuJiOK8iqPurNXW/yQfEH3Dv9XlKjdJmMUipwgR6Gesn3tog8D7zlSETKEZV1lTy49kEmDJnAwrEL3Q5HKdXPdLViXA6Q1ZOBKGf9+uNfc/jYYR45/RGCg4LdDkcp1c8EOmZRScsxiwNY57hQ/cDmQ5tZ8dkKLhl3CbnJuW6Ho5TqhwI9DBXrdCDKGY2eRn7y4U8YEjmEWyff6nY4Sql+KqDaUCJysYjE+9xOEJGLnAtL9ZQVn69ga+lW7p52N7FhmvOVUl0TaCHBHxtjyptuGGOOAD92JiTVU0qqS/j1x79mVvoszsk+x+1wlFL9WKDJor12ejq1Pu7na39OfWM9P5jxAy0UqJTqlkCTRaGIPCQio0VklIj8EljnZGCqe/6z7z/8fdffuW7idWTF6cQ1pVT3BJosbgXqgBeAPwPHgJudCkp1T01DDfevuZ/suGyuztNF9kqp7gt0NlQVcI/Dsage8uSnT7K3ci9Pnv0kYcFhboejlBoAAp0NtUpEEnxuJ4rIm86FpbpqZ/lOfr/p98wbNY8Zw2a4HY5SaoAI9DBUsj0DCgBjTBl6Du4+xxjD/avvJzIkku8WfNftcJRSA0igycIjIt5RUhHJpp0qtMpdr+98nY8OfMTtU24nOTLZ7XCUUgNIoNNffwD8W0T+ad/+GnC9MyGpriivLecXhb9gYspEFpywwO1wlFIDTKAD3H8XkQKsBLEBeA1rRpTqIx5e/zDlteU8cdYTBEmgHUallApMoIUErwVuAzKxksVM4ENanmZVuWRDyQb+8sVfuHzC5YxNGut2OEqpASjQr6C3AdOA3caY04DJwEHHolIBa/A0cN/q+0iLSuOm/JvcDkcpNUAFmixqjDE1ACISboz5DPD7FVZE5orI5yKyXUTaXachIt8UkS0isllE/uSzvVFENtiXlQHGOeg8t/U5vij7gnun30t0aLTb4SilBqhAB7iL7HUWrwKrRKQMP6dVFZFg4FHgLKAIWCsiK40xW3za5AD3ArONMWUi4jsd95gxJr8Tr2XQOVB1gEc3PMqpmadyepYeEVRKOSfQAe6L7av/KyLvAvHA3/3sNh3YbozZCSAiK4D5wBafNtcBj9rrNjDGlHQi9kFv6UdLMcZw74x7tVCgUspRnZ42Y4z5pzFmpTGmzk/TDGCvz+0ie5uvE4ATROQ/IrJaROb63BchIoX2dj13Rivv7X2Pt/e8zY2TbiQjpvXbqpRSPcvJMuPtfdVtvZAvBOt83nOwZlr9S0Ty7NXiWcaYYhEZBbwjIp8aY3a0eAKR67HXe2RlDZ7KqtX11Tyw5gHGJIzh8tzL3Q5HKTUIODkhvwgY7nM7k7bjHEXAa8YQRkzkAAAbnUlEQVSYemPMl8DnWMkDY0yx/XMn8B7WDKwWjDFPGGMKjDEFKSkpPf8K+qjHNz5OcVUxi2YuIjQo1O1wlFKDgJPJYi2QIyIjRSQMuARoPavpVeA0ABFJxjostdMuVBjus302Lcc6Bq1tZdt4ZvMzXDzmYqamTXU7HKXUIOHYYShjTIOI3AK8CQQDy40xm0VkMVBojFlp33e2iGwBGoG7jDGHRWQW8LiIeLAS2lLfWVSDlcd4WLJ6CTFhMdwx9Q63w1FKDSKOnhrVGPMG8EarbT/yuW6AO+2Lb5sPgBOdjK0/em37a6wvWc/iWYtJjEh0Oxyl1CCiRYT6ibKaMpatW8aU1CnMHzPf7XCUUoOMJot+4qF1D1FVV8WimYu0UKBSqtfpp04/UHigkFe3v8rluZeTk5jjdjhKqUFIk0UfV99Yz5LVS0iPTueGiTe4HY5SapBydIBbdd/TW55mR/kOHjn9EaJCo9wORyk1SGnPog8rqizi8U8e54ysMzh1+Kluh6OUGsQ0WfRRxhge+OgBRIR7prdb3V0ppXqNJos+6p097/B+0fvcnH8zQ6OHuh2OUmqQ02TRB1XVV/HARw8wNnEs3x7/bbfDUUopHeDui36z4TeUVJewbM4yQoL0V6SUcp/2LPqYz0o/47mtz7HghAVMSpnkdjhKKQVosuhTPMbDfR/eR3x4PLdNuc3tcJRSykuTRR/y4hcvsvHQRr5X8D3iw+PdDkcppbw0WfQRh44d4lfrf8X0odOZN2qe2+EopVQLmiz6iGWFy6hpqGHRzEWItHdGWqWUco8miz5gzf41vL7zda7Ou5qR8SPdDkcppdrQZOGyusY6lqxewvDY4Vx74rVuh6OUUu3SSfwuW75pObsqdvHbM39LREiE2+EopVS7tGfhoj0Ve/jdxt8xN3suszNmux2OUkp1SJOFS4wx3L/mfsKCw7hr2l1uh6OUUselycIlb+56kw+KP+DWybeSGpXqdjhKKXVcmixcUFlXyYNrH2TCkAksHLvQ7XCUUsovHeB2wa8//jWlNaU8csYjBAcFux2OUkr5pT2LXrb50GZWfLaChWMXkjsk1+1wlFIqIJoselGjp5GffPgThkQO4dbJt7odjlJKBUyTRS9a8fkKtpZu5e5pdxMbFut2OEopFTAds+glJdUl/PrjXzMrfRbnZJ/jdjiqj6uvr6eoqIiamhq3Q1EDREREBJmZmYSGhnZpf00WveRna39GfWM9P5jxAy0UqPwqKioiNjaW7Oxs/XtR3WaM4fDhwxQVFTFyZNfqzzl6GEpE5orI5yKyXUTu6aDNN0Vki4hsFpE/+Wy/QkS22ZcrnIzTaf/e92/e3PUm1028jqy4LLfDUf1ATU0NQ4YM0USheoSIMGTIkG71VB3rWYhIMPAocBZQBKwVkZXGmC0+bXKAe4HZxpgyEUm1tycBPwYKAAOss/ctcypep9Q01HD/6vvJjsvm6ryr3Q5H9SOaKFRP6u7fk5M9i+nAdmPMTmNMHbACmN+qzXXAo01JwBhTYm8/B1hljCm171sFzHUwVsf87tPfUXS0iEUzFxEWHOZ2OEp12owZM8jPzycrK4uUlBTy8/PJz89n165dnXqcl19+mc8++6zTz3/yySezYcOGTu/X5Be/+AV/+tOf/Dd00Te+8Q127tzZ7n0lJSXMmTOH6Ohobr/99g4f4/Dhw5xxxhnk5ORwzjnnUF5e3qMxOpksMoC9PreL7G2+TgBOEJH/iMhqEZnbiX0RketFpFBECg8ePNiDofeMneU7Wb5pOfNGzWPGsBluh6NUl6xZs4YNGzawePFiFi5cyIYNG9iwYQPZ2dmdepyuJovuqK+v59lnn2Xhwr5dKeHGG2/k5z//ebv3RUVFcf/99/Pggw8e9zHuv/9+zj33XLZt28Ypp5zCz372sx6N0clk0V6fx7S6HQLkAHOAS4EnRSQhwH0xxjxhjCkwxhSkpKR0M9yeZYxhyeolRIZE8t2C77odjlKO+Nvf/sZJJ53ElClTWLhwIVVVVQDcddddTJgwgYkTJ3L33Xfzr3/9izfeeIM77rijS72SJn/84x858cQTycvL4/vf/753++OPP84JJ5zAnDlzuPbaa73fwFetWsW0adMIDrYqJaxevZqJEycya9Ys7rrrLvLz8wHYsWMHp5xyCpMnT2bq1KmsWbMGgLfeeovTTjuNBQsWkJOTw6JFi3jmmWeYNm0aEydO9L6Oyy67jJtvvpnTTjuN0aNH8/7773PFFVcwbtw4rrnmGm+c119/PQUFBeTm5rJ48WLv9jlz5vD3v/+dxsbGNq85JiaG2bNnExFx/FMYvPbaa1xxhTW8e8UVV/Dqq6929u09LidnQxUBw31uZwLF7bRZbYypB74Ukc+xkkcRVgLx3fc9xyJ1wOs7X2ftgbX8cOYPSY5Mdjsc1Y/95K+b2VJc0aOPOSE9jh9f0L0KAiUlJSxdupS3337b++334Ycf5pprruGNN95g8+bNiAhHjhwhISGB8847jwULFnDRRRd16fmKiopYtGgRhYWFxMfHc+aZZ/L6668zadIkli5dyvr164mOjmbOnDlMnz4dgP/85z9MnTrV+xhXXXUVTz/9NNOnT+d73/ued/uwYcNYtWoVERERfPbZZ1xxxRXehPHJJ5+wdetW4uPjyc7O5qabbmLt2rUsW7aMRx55hF/84hcAlJeX8+677/LSSy9xwQUX8OGHHzJu3DimTJnCpk2byMvLY+nSpSQlJdHQ0OBNQhMmTCA4OJjs7Gw2bdrEpEmTuvT+HD58mKYvzRkZGezfv79Lj9MRJ3sWa4EcERkpImHAJcDKVm1eBU4DEJFkrMNSO4E3gbNFJFFEEoGz7W39QnltOb8o/AUTUyay4IQFboejlCM++OADtmzZwqxZs8jPz+e5555j165dJCUlERQUxHXXXccrr7xCdHR0jzzfmjVrOP3000lOTiY0NJRvfetbvP/++97tiYmJhIWFsWBB8//c/v37vR+ghw4doq6uzptIvvWtb3nb1dbWcs0115CXl8cll1zCli3eeTjMmDGDtLQ0IiIiGDVqFOecY62TOvHEE1v0kC644ALv9vT0dCZMmEBQUBATJkzwtnv++eeZMmUKU6ZMYevWrS2eJzU1leLi1t+nu66nJ0g41rMwxjSIyC1YH/LBwHJjzGYRWQwUGmNW0pwUtgCNwF3GmMMAInIfVsIBWGyMKXUq1p728PqHKa8t54mzniBIdJG86p7u9gCcYoxh7ty5PPvss23uKywsZNWqVaxYsYLHHnuMf/zjHx0+ju8H+Ne//nV+9KMfdfh8ndkOEBkZ6Z0uerx2y5YtY/jw4fzxj3+kvr6emJgY733h4eHe60FBQd7bQUFBNDQ0tGnn28a33bZt23j44Yf56KOPSEhI4LLLLmsxlbWmpobIyEhefPFFlixZAsBTTz3lPVTmz5AhQzh48CApKSns27ePoUOHBrRfoBz9JDPGvGGMOcEYM9oYc7+97Ud2osBY7jTGTDDGnGiMWeGz73JjzBj78gcn4+xJG0o28Jcv/sK3x3+bsUlj3Q5HKcfMmjWLf/7zn95ZPFVVVWzbto3KykoqKiqYN28ev/zlL/n4448BiI2NpbKyss3jhIWFeQfNO0oUADNnzuTdd9/l8OHDNDQ0sGLFCk499VRmzJjBu+++y5EjR6ivr+fll1/27jN+/Hi2b98OQEpKCqGhoRQWFgKwYoX344by8nKGDRuGiPD0008fN7F0VUVFBbGxscTFxbF//37efLPlwZJt27aRm5vLggULvO9HoIkC4MILL+Tpp58G4Omnn2b+/NaTT7tHV3D3oAZPA/etvo+0qDRuyr/J7XCUclRaWhq///3vWbhwIXV1dQD89Kc/JTIykq9//evU1tbi8Xh46KGHALj00ku54YYbWLZsGa+++mqnZ1NlZmayePFi5syZgzGGCy64gPPPPx+wBtSnT59ORkYGubm5xMfHA3Deeee1GGBevnw5V111FbGxsXzta1/ztrvllltYsGABzz//PGeeeWaLnkFPmTJlChMmTCAvL49Ro0Yxe3bzqZSLi4uJj4+no4k6mZmZVFdXU19fz4svvsjbb7/N2LFjueqqq7jtttvIz8/n+9//Pt/85jd5/PHHGTlyJC+88ELPvgBjzIC4TJ061bjtqU1Pmbyn8sxbu95yOxTVz23ZssXtEPqVyspKY4wxdXV15txzzzUrV6703nfBBReYHTt2tGhnjDFLliwxd955Z+8G2oGf/exn5qmnnnL8edr7u8IaFvD7GasH1HvIgaoDPLrhUU7NPJXTs053OxylBpUf/vCHTJ48mYkTJzJ27FjmzZvnve/BBx/0DhyvXLmS/Px88vLy+PDDD7n33nvdCrmFIUOGcNlll7kdxnGJceDYnBsKCgpM07FIN9z+7u38Z99/ePWiV8mIabN+UKlO2bp1K+PHj3c7DDXAtPd3JSLrjDEF/vbVnkUPeG/ve7y9521unHSjJgql1ICkyaKbquureWDNA4xJGMPluZe7HY5SSjlCZ0N10+MbH6e4qpin5j5FaFDXTiqilFJ9nfYsumFb2Tae2fwMF4+5mKlpU/3voJRS/ZQmiy7yGA9LVi8hJiyGO6be4XY4SjlGS5Q7r3WJ8rVr15KXl8eYMWO44472P1+MMdx0002MGTOGSZMmdes9CoQmiy56bftrrC9Zz51T7yQxItHtcJRyjJYod17rEuU33ngjf/jDH9i2bRubN29m1apVbfb561//yt69e9m+fTuPPvooN998s6MxarLogrKaMpatW8aU1CnMH9OzS+qV6k+0RLn1OnqyRPnevXupqalh2rRpiAjf+c532i03/tprr3H55dakmpNPPpkDBw7g5Hl9dIC7Cx5a9xBVdVUsmrlICwUq5/3tHjjwac8+5tAT4dyl3XoILVHuTInyY8eOMXx489kdMjMz2bdvX5v3Y9++fe22c+rcPvpJ10mFBwp5dfurXJ57OTmJOW6Ho5RrtES5MyXK21so3V658UDb9RTtWXRCfWM9S1YvIT06nRsm3uB2OGqw6GYPwClGS5Q7UqI8MzOTvXubzypdVFREenp6m5ib2s2cOfO47XqK9iw64ektT7OjfAffn/F9okKj3A5HKVdpifLOCbRE+fDhwwkPD2ft2rUYY3j22WfbLTd+4YUX8swzzwDw73//m7S0NMcOQYH2LAJWVFnE4588zhlZZ3Dq8FPdDkcp12mJ8s7pTInyxx57jCuvvJKamhrmzZvHWWedBcCjjz5KeHg41157LRdccAF/+9vfGD16NNHR0d5zWTgmkNK0/eHiZIlyj8dj/nvVf5tpf5xm9h/d79jzKNVES5R3jpYoD4yWKHfY23ve5l/7/sXN+TczNLpnT1WolOo+LVHuPC1R7kdVfRUXvnohieGJrJi3gpAgPXKnnKclypUTulOiXD/5/Hh0w6McrD7IQ3Me0kShlBq09DDUcWw9vJXntj7HghMWMCllktvhKKWUazRZdKDR08h9q+8jITyB26bc5nY4SinlKk0WHXhp20t8euhTvlfwPeLD490ORymlXKXJoh2Hjh3iV+t+xfSh05k3ap7/HZQawLREufNalyi/5557yMzMJCEh4bj7LVmyhDFjxjBu3DjeeustR2PUEdt2LCtcRk1jDYtmLnK01opS/UFTQb2nnnqKwsJCHnnkkS49zssvv0xQUBDjxo3ryfCOq6lE+fr163vtObuiqUT5Y489BsD8+fO55ZZbyMvL63CfjRs38vLLL7Nlyxb27t3L3Llz+fzzzwkKcqYPoD2LVtbsX8PrO1/n6ryrGRk/0u1wlOrTtES59Tp6skQ5wEknncTQocdf0/Xaa69x6aWXEhYWxujRo8nKymLdunVdel8DoT0LH3WNdSxZvYThscO59sRr3Q5HKQAe/OhBPivt2ZMGjUsax93T7+7WY2iJcmdKlE+aFNjMy3379jFnzhzv7aYS5dOmTevS++uP9ix8LN+0nF0Vu/jBjB8QERLhdjhK9WlaotyZEuWBam9Bdb8tUS4ic4GHgWDgSWPM0lb3Xwn8HGg6s8cjxpgn7fsagaYzvuwxxlzoZKx7Kvbwu42/Y272XGZnzPa/g1K9pLs9AKcYLVHuSInyQAVayrynONazEJFg4FHgXGACcKmITGin6QvGmHz78qTP9mM+2x1NFMYY7l9zP2HBYdw17S4nn0qpAUNLlHdOoCXKA3XhhRfy/PPPU1dXx44dO9i9e3eLQ249zcnDUNOB7caYncaYOmAF0CdPWP3mrjf5oPgDbp18K6lRqW6Ho1S/4FuifNKkScyaNYsvvviC8vJyzj//fCZNmsTpp5/eokT5T3/60y4PcPuWKM/Pz2fmzJmcf/75ZGVleUuUn3322W1KlP/zn//0PkZTifJZs2YRFBTUokT5k08+ycyZM9m9e7fjJcqvu+6645Yov/POO8nOzqaiooLMzEyWLFkCwCuvvOIdGJ80aRIXXXQR48eP57zzzuM3v/mNYzOhAOdKlAMLsA49Nd3+DtZhJt82VwL7gY3Ai8Bwn/sagEJgNXBRB89xvd2mMCsrqysVe01FbYWZ88Ic882/ftM0NDZ06TGU6mlaorxztER5YPpqifL2Rlpa9+3+CmQbYyYCbwG+Z+/IMlYlxG8BvxKR0W0ezJgnjDEFxpiCrp4hqraxlhOTT+RHM39EcFBwlx5DKeUuLVHuPMdKlIvIScD/GmPOsW/fC2CMeaCD9sFAqTGmTW0NEXkKeN0Y82JHz+dUiXKl3KAlypUTulOi3MmexVogR0RGikgYcAmw0reBiAzzuXkhsNXenigi4fb1ZGA2sAWllFKucGzqrDGmQURuAd7Emjq73BizWUQWYx0jWwn8j4hciDU+UYo1hgEwHnhcRDxYCW2pMUaThRpUjDFabkb1mO4eRdIz5SnVB3355ZfExsYyZMgQTRiq24wxHD58mMrKSkaObFnGSM+Up1Q/lpmZSVFREQcPHnQ7FDVAREREkJmZ2eX9NVko1QeFhoa2+QaolJu0NpRSSim/NFkopZTyS5OFUkopvwbMbCgROQjs7sZDJAOHeiicnqRxdY7G1TkaV+cMxLhGGGP8lsAYMMmiu0SkMJDpY71N4+ocjatzNK7OGcxx6WEopZRSfmmyUEop5Zcmi2ZPuB1ABzSuztG4Okfj6pxBG5eOWSillPJLexZKKaX8GrTJQkS+ISKbRcQjIh3OIhCRuSLyuYhsF5F7eiGuJBFZJSLb7J+JHbRrFJEN9mVle216KJ7jvn4RCReRF+z714hItlOxdCKmK0XkoM/7c63TMdnPu1xESkRkUwf3i4j8f3bcG0VkSh+Ja46IlPu8Xx2fCLtn4xouIu+KyFb7f/G2dtr0+nsWYFy9/p6JSISIfCQin9hx/aSdNs79PwZyOr2BeMEqgz4WeA8o6KBNMLADGAWEAZ8AExyO62fAPfb1e4AHO2h3tBfeI7+vH7gJ+K19/RLghT4Q05W0OoVvL/1NfQ2YAmzq4P7zgL9hnUVyJrCmj8Q1B+vkYr39fg0DptjXY4Ev2vld9vp7FmBcvf6e2e9BjH09FFgDzGzVxrH/x0HbszDGbDXGfO6n2XRguzFmpzGmDlgBzHc4tPk0n172aeAih5/veAJ5/b7xvgicIc7W1HbjdxIQY8z7WOdl6ch84BljWQ0ktDoBmFtxucIYs98Ys96+Xol18rOMVs16/T0LMK5eZ78HR+2bofal9aCzY/+PgzZZBCgD2Otzuwjn/2jSjDH7wfqjBVI7aBchIoUislpEnEoogbx+bxtjTANQDgxxKJ5AYwL4L/uwxYsiMtzBeDrDjb+nQJ1kH974m4jk9vaT24dLJmN9W/bl6nt2nLjAhfdMRIJFZANQAqwyxnT4fvX0/+OALlEuIm8BQ9u56wfGmNcCeYh2tnV7+tjx4urEw2QZY4pFZBTwjoh8aozZ0d3YWgnk9TvyHh1HIM/3V+B5Y0ytiNyI9U3rdAdjClRvv1eBWo9V8uGoiJwHvArk9NaTi0gM8BJwuzGmovXd7ezSK++Zn7hcec+MMY1AvogkAK+ISJ4xxncsyrH3a0AnC2PMmd18iCLA91tpJlDczcc8blwi8pWIDDPG7Le72yUdPEax/XOniLyH9e2np5NFIK+/qU2RiIQA8Th7yMNvTMaYwz43fwc86GA8neHI31N3+X4QGmPeEJHfiEiyMcbxGkgiEor1gfycMebldpq48p75i8vN98x+ziP2//1cwDdZOPb/qIehjm8tkCMiI0UkDGvAyLGZR7aVwBX29SuANj0gEUkUkXD7ejIwG3DiHOWBvH7feBcA7xh7dM0hfmNqdUz7Qqxjzn3BSuBye4bPTKC86ZCjm0RkaNNxbRGZjvW5cPj4e/XI8wrwe2CrMeahDpr1+nsWSFxuvGcikmL3KBCRSOBM4LNWzZz7f+zN0fy+dAEuxsrCtcBXwJv29nTgDZ9252HNhtiBdfjK6biGAG8D2+yfSfb2AuBJ+/os4FOsmUCfAtc4GE+b1w8sBi60r0cAfwG2Ax8Bo3rhPfIX0wPAZvv9eRcY10t/U88D+4F6+2/rGuBG4Eb7fgEeteP+lA5m4bkQ1y0+79dqYFYvxXUy1iGSjcAG+3Ke2+9ZgHH1+nsGTAQ+tuPaBPzI3t4r/4+6glsppZRfehhKKaWUX5oslFJK+aXJQimllF+aLJRSSvmlyUIppZRfmiyU6gQROeq/1XH3f9FedY+IxIjI4yKyw64i+r6IzBCRMPv6gF40q/oXTRZK9RK7flCwMWanvelJrNW1OcaYXKxqucnGKpD4NrDQlUCVaocmC6W6wF5R/HMR2SQin4rIQnt7kF36YbOIvC4ib4jIAnu3b2OvyBeR0cAMYJExxgNW6RZjzP/ZbV+12yvVJ2g3V6mu+TqQD0wCkoG1IvI+VumVbOBErIrBW4Hl9j6zsVZTA+QCG4xVGK49m4BpjkSuVBdoz0KprjkZq7JtozHmK+CfWB/uJwN/McZ4jDEHsMqNNBkGHAzkwe0kUicisT0ct1JdoslCqa7p6IQyxzvRzDGs2j1g1RWaJCLH+x8MB2q6EJtSPU6ThVJd8z6w0D4ZTQrWqUs/Av6NdeKlIBFJwzr9ZpOtwBgAY517pBD4iU/10hwRmW9fHwIcNMbU99YLUup4NFko1TWvYFX//AR4B/h/9mGnl7Aqu24CHsc6w1q5vc//0TJ5XIt1EqztIvIp1rk3ms7VcBrwhrMvQanAadVZpXqYiMQY6wxqQ7B6G7ONMQfscxC8a9/uaGC76TFeBu41/s8Tr1Sv0NlQSvW81+2T1IQB99k9Dowxx0Tkx1jnSd7T0c72SZ1e1USh+hLtWSillPJLxyyUUkr5pclCKaWUX5oslFJK+aXJQimllF+aLJRSSvmlyUIppZRf/z9HSj2AamWDPQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#分析参数C和gamma对模型复杂度的影响，以及对训练误差和交叉验证得到的测试误差的影响。\n",
    "Otto_SVM_result = pd.read_csv(\"Otto_SVM_result.csv\")\n",
    "accuracy_s1 = Otto_SVM_result['accuracy']\n",
    "\n",
    "C_s = np.logspace(-1, 3, 5) \n",
    "gamma_s = np.logspace(-1, 1, 3)  \n",
    "accuracy_s1 =np.array(accuracy_s1).reshape(len(C_s),len(gamma_s))\n",
    "\n",
    "x_axis = np.log10(C_s)\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    plt.plot(x_axis, np.array(accuracy_s1[:,j]), label = ' Test - log(gamma)' + str(np.log10(gamma)))\n",
    "\n",
    "plt.legend()\n",
    "plt.xlabel( 'log(C)' )                                                                                                      \n",
    "plt.ylabel( 'accuracy' )\n",
    "plt.savefig('RBF_SVM_Otto.png' )\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best_C: 10.0\n",
      "Best_gamma: 1.0\n"
     ]
    }
   ],
   "source": [
    "#得到最佳超参数\n",
    "index = np.unravel_index(np.argmax(accuracy_s1, axis=None), accuracy_s1.shape)\n",
    "Best_C = C_s[ index[0] ]\n",
    "Best_gamma = gamma_s[ index[1] ]\n",
    "print(\"Best_C:\",Best_C)\n",
    "print(\"Best_gamma:\",Best_gamma)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "250.28px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
